Observer Variability: A New Approach in Evaluating Interobserver Agreement

Existing indices of observer agreement for continuous data, such as the intraclass correlation coefficient or concordance coefficient, measure total observer-related variability, which includes variabilities between and within observers. This work introduces a new index that measures interobserver is defined in terms distances among ‘true values’ assigned by different observers on same subject. The variability (CIV ) ratio variability. We show how to estimate CIV use bootstrap ANOVAbased methods inference. also develop excess compares expected when there are no differences simple function . In addition, we value , estimated from an study, can be used design measurements studies. illustrate concepts two examples, where (1) radiologists calcium scores evaluate severity coronary artery arteriosclerosis, (2) were knee joint angle.